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Abstract 


Augmented reality (AR) is a challenging technology for e-learning. Nevertheless, the benefits of AR in terms 
of increased engagement and motivation could be undermined by a low ergonomic quality. A challenge 
for designers is to provide students with a usable AR environment that is able to support an effective and 
efficient achievement of their learning goals. A user-centred design approach requires understanding 
users and the tasks they perform. This analysis focuses on task modelling and presents a task-based 
approach to the development of a Chemistry application implemented on an AR teaching platform. The 
task model enables an analysis of critical task sequences, helps to prevent potential usability problem, 
and provides with a basis for a task-based approach to evaluation. 
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reality. 
Introduction 


The ISO standard 25010 distinguishes between a product quality model and a quality in 
use model (ISO, 2010). The quality in use is closely related with five outcomes of interaction 
with the system: effectiveness, efficiency, satisfaction, safety, and context comprehensiveness. 
The product quality model defines eight characteristics: functional suitability, performance 
efficiency, compatibility, usability, reliability, security, maintainability, and portability. Usability 
refers to the extent to which a product can be used by specified users to accomplish specified 
goals with effectiveness, efficiency, and satisfaction in a specified context of use (ISO, 2010). 
At a more detailed level, usability refers to products that are recognized as appropriate, are 
easy to learn how to use, easy to operate, protect users from errors, and have an aesthetic user 
interface. 

Augmented reality integrates real and virtual worlds (computer generated images) and 
is featuring a real time 3D interaction targeting all senses (Azuma, 1997). AR technologies are 
creating new opportunities for the designers of e-learning systems (Cheng & Tsai, 2013; Wu 
et al., 2013). By integrating various real objects from the traditional school into a computing 
environment, AR favours learning by doing which is more captivating and motivating, especially 
for the young learner (Vos et al., 2011; di Serio et al, 2013). The ergonomic quality is a key 
factor influencing the outcomes of an AR-based application (Dunser & Billighurst, 2011; Bai 
& Blackwell, 2012; Pribeanu, 2014). By ergonomic quality we refer to the extent to which a 
system is easy to understand, easy to learn how to use, and easy to operate. The augmented 
reality technology may create additional problems that are due to the specific devices (see- 
through screen, projectors, and cameras) as well as to the integration of specific real objects. 
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In general, designing for usability is not easy for applications based on augmented reality and 
requires a user-centred approach to design and evaluation (Bach & Scapin, 2004; Gabbard & 
Swann, 2008). This means to understand users and tasks, to carefully develop specific interaction 
techniques, and to evaluate with actual users. 

The ARiSE (Augmented Reality in School Environments) project created an Augmented 
Reality Teaching Platform (ARTP) for primary and secondary schools. The main goal was to 
test the pedagogical effectiveness of using the AR technology in class. A specific objective was 
to test the extent to which ARTP 1s enhancing students’ motivation to learn. Three application 
prototypes were developed onto ARTP. The second application implemented a Chemistry 
learning scenario. The underlying interaction paradigm for this scenario was “building with 
guidance” aiming at understanding the periodic table of Chemical elements, the structure of 
atoms / molecules, and the chemical reactions. 

In order to ensure a user-centred approach, a task model has been developed for each 
application prototype. The objective of this paper is to present the task-based design approach 
and to highlight the benefits of task modelling for the ergonomic quality of the application. 
The approach is illustrated with an example (the task of building molecules and Chemical 
compounds from atoms). 

The rest of this paper 1s organized as follows. The platform and experiment are briefly presented 
in the next section. Then the notation used for task modelling and some previous and related 
work in this area are summarized. Then the task modelling approach is presented and discussed. 
The paper ends with conclusion and future research directions. 


Platform and Learning Scenario 


The AR platform consists of 4 independent modules organized around a table on which 
real objects are placed (Wind et al., 2007). The platform has been registered by Fraunhofer [AIS 
(Spinnstube®). The Chemistry scenario is using two kinds of real objects: a periodic table and a 
set of coloured balls (4 colours) symbolizing atoms. A remote controller Wii Nintendo has been 
used as interaction tool for selecting a menu item. 





Figure 1: Student performing an exercise in Lesson 2. 


The learning scenario for Chemistry has an introductory part and three lessons. The first 
lesson is about the chemical structure of the atoms and has 2 exercises. The second lesson is 
about forming molecules and has 8 simple exercises. The third lesson 1s about chemical reactions 
and has 3 exercises. Each lesson has several exercises. Additional details about the pedagogical 
goals, lessons, and exercises could be found in (Vilkonis et al., 2008). Each workplace has its 
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own periodic table and a set of 24 balls. By placing a ball over a chemical element it becomes 
an atom of that element. Then atoms could be further used to build molecules and Chemical 
compounds. 


Task Modelling Notations and Tools 


Task modelling is an important concern for developers aiming at producing usable 
systems. A task is an activity performed by people to accomplish a certain goal. A task could 
be further decomposed resulting in smaller tasks corresponding to lower level goals. Task 
decomposition is usually represented as a tree. Inner tasks are said to be composite (or abstract, 
complex) while the leaves are elementary tasks, which in turn are decomposed in actions 
performed upon objects. Task models as resulted from early task analysis could provide with 
useful information for improving user guidance and task compatibility. 
In a user centred design approach the user interface of an interactive system is designed based 
on the information provided by the task model. Existing task models show a great diversity 
as regarding formalisms and depth of analysis. There are various objectives that underlie the 
elaboration of each task model thus featuring a certain type of representation (Pribeanu & 
Vanderdonckt, 2002): 
e Inform design about potential problems of usability, as in HTA (Annett & Duncan, 
1967). 

8 Evaluate the human performance, as in GOMS (Card et al., 1983). 

° Aid design by providing a more detailed task model describing task hierarchy, 
objects used and knowledge, as in TKS (Johnson et al., 1992) or GTA (van der 
Veer et al., 1996). 

e Generate a prototype of a user interface, as in Adept (Wilson et al., 1993). 


e In a previous work (Pribeanu, 2005) a task modelling approach for user interface 
design has been proposed that is based on three decomposition levels: 

© A functional level that results from mapping of application functions onto user 
tasks. 

° A unit task level that results from the decomposition of functional tasks regardless 
the constraints imposed by a target hardware and software platform (goal 
hierarchy). 

e A basic task level that results from the decomposition of unit tasks, which 


are, carried on by using interaction techniques available on a target platform 
(operational task structure). 

© A unit task is defined as a task the user really wants to perform (Card et al, 
1983). The distinction between unit tasks and basic tasks is important for at least 
two reasons: (a) it follows the design process, by separating the “what-to-do” 
knowledge acquired during task analysis from the “how-to-do-it” knowledge 
acquired during task design; (b) basic tasks are pointing to abstract interaction 
objects thus rendering visible in the task model in the interaction flow. 

© In a recent work, Molina et al. (2013) proposed the Collaborative Interactive 
Application Notation — CIAN and a metamodel for an associated modelling tool 
supporting interactive group work. Their conceptual framework distinguishes 
between four modelling perspectives: organizational (organizational structure), 
process (work structure and work flow), data (information manipulated), and 
interaction (interaction with the presentation layer). From a process modelling 
perspective, they consider individual tasks, group work tasks, collaborative tasks, 
and cooperative tasks. 
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@ There are relatively few approaches to task modelling for AR-based applications. 
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As pointed out by Trevisan et al. (2004) it is more difficult to model tasks in 
the context of two worlds of interaction (real and virtual) and multiple sources 
of information. They suggested an integration of several notations in order to 


express the specific task characteristics in AR. 


e Dupuy-Chessa et al. (2010) proposed a software engineering method using the 
ASUR notation for task modelling. They distinguish between abstract scenarios 
and concrete scenarios. A concrete scenario describes the user tasks, the supporting 
devices, and the interaction artefacts. The scenario is checked for usability based 


on paper and software prototypes. 


© Several notations and tools exist for task modelling. Concurrent Task Trees - CTT 
(Paterno & Santoro, 2002) is a notation that is using temporal operators, to link 
sibling tasks. This is different from other notations where operators are describing 
parent-children relationship. CTTE is an interactive environment providing a 
graphical notation to specify tasks, roles, and objects as well as the task hierarchies. 
There are five kinds of task categories: abstract, interaction, application, user, and 
cooperative. The task categories are presented in Table 1. 


Table 1. Task categories in CTTE. 


Abstract Interaction Application 


2 fe 5. 


Cooperative 


i) 


The temporal operators defined by CTT are presented in Table 2. The notation 
distinguishes between unary and binary operators. There are several restrictions in combining 
binary and unary operators. For example, the combination T1*>>T2 is not allowed, since T2 
will be never performed. Also, optional tasks are not allowed in the left and right side of the 


operators |>, [> and [ ]. 


Table 2. Temporal operators in CTTE. 


Binary operators 
Choice 


Order independency 


Interleaving 


Synchronization 


Enabling 
Enabling with info passing 
Disabling 
Suspend / resume 
Unary operators 
Optional 


Iteration 
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T1 []T2 
T1 [=| 2 
T1 ||| T2 
T1 [| 72 
T1 >> T2 
T1 [>> T2 
T1 [> T2 
T1 |> T2 
[T1] 
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With CTTE a designer can create task trees and specify task properties such as: task type, 
frequency, and estimated execution time. An important feature is the XML output capability 
that makes CTTE a useful tool for the handling of mappings between the task model and other 
models. An example is given in (Pribeanu, 2006) where a set of mapping rules between task 
and domain models has been used for automate derivation of task model fragments from the 
domain model. 

A first approach to task modelling in the ARiSE project was taken for the Biology 
learning scenario. The idea was to model each task by using the CTT notation and to link 
a tabular description for each task model with the graphical representation. The approach is 
described in a previous work (Pribeanu et al., 2007). 


Task-based Approach 


In order to present our task-based approach we will take as an example an exercise from 
the second lesson of the Chemistry scenario. The task goal is to build a molecule from atoms. 
The description for the second lesson is summarized in Table 3 (at unit task level) and is based 
on the scenario defined by Vilkonis et al. (2008). 

At the beginning of each lesson the student is given a vocal explanation on how to 
perform the tasks. Then, a list of exercises is displayed from which the student can select one 
by pressing the back button of the remote control. A short explanation is given for the exercise 
and the Chemical symbol of the molecule is displayed on the screen. Then s (he) takes a ball 
and places it on the periodic table. After watching the chemical structure and checking the 
chemical symbol of the atom (visual feedback) the student confirms by pressing the back button 
of the remote controller. The task is repeated for each atom required. Then the student brings 
the atoms together to build the molecule. If the result is correct, (s) he receives a positive audio 
feedback and the symbol is highlighted in green. 


Table 3. Scenario description for an exercise in Lesson 2. 


Task name User input System output Description / observations 


Select the exercise Vocal explanation, displays the Select the exercise by pressing the 


ROI EN eIERS from the menu symbol of the molecule back button of the remote control 


Place aballonthe — Displays the atom structure and Iterative abstract task, to be 


ee rcrelan table and confirm atom symbol: waits for confirmation poe HNL AN eeCe alates 
are specified 
If the result is correct, displays the Iterative user task, to be performed 
. Place all atoms on . 
Build molecule augmentation, symbol; vocal mes- until all atoms needed are brought 
the table 
sage to congratulate the student together 
Quit Select “Quit” from Shinn foie icharencreices Can repeat the exercise or select 


the menu a new one 


Prior to build the task model the scenario is analysed for usability. Good ergonomic 
quality requires user guidance throughout the interaction process. Appropriate user guidance 
requires prompting the user for input and providing adequate feedback. As the screen space 1s a 
critical resource for desktop AR systems, providing with vocal output (prompting, explanations, 
and feedback) is very useful. 

The task model is specified in CTTE according to the scenario description. There are 
four unit tasks: Exercise, Select atom, Build molecule, and Quit. The model could be checked 
in CTTE for consistency. This way, it is possible to detect tasks that are not reachable. Then, 
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it is possible to simulate the task execution which is useful both for designers (check if task 
implementation is consistent with the scenario requirements) and evaluators. The simulated 
scenario (tasks actually performed) could be saved for later use. Prior to evaluation, it 1s 
useful to understand task requirements and check if the user input and system output are well 
synchronized. The task model for Lesson 2 is presented in Figure 2. 





place ball displaysymbol display structure coin bring atoms together* display augmentation oe a congratulation 


Figure 2: Task model for Lesson 2. 


In order to inform designers on potential usability problems, the operational task 
structure for each unit task should be carefully analysed. In Table 4 a template is presented that 
is intended to guide designers for usability. The description is done at unit task level. The same 
template could be used by evaluators to prepare the evaluation and to identify actual usability 
problems. 


Table 4. Potential usability problems — Select atom. 


Basic Task Potential problem Solution 


Vocal explanation at unit task level: attention to use balls with 


Select atom Lack of user guidance different colour for each atom and confirm 


Vocal feedback if the same colour has already been assigned 


Place ball Same colour for more atoms 
(warning) 


The ball ts placed on semantic feedback: augmentation showing the atom structure 


another element 

Display symbol Same colour for more atoms _ Visual feedback: atom symbol — coloured ball (prevent) 
The user does not perceive Vocal feedback on where the symbol is displayed (e.g. top-right 
it area) 

Display structure Visual perception Augmentation should be accurate 

Pantin Wrong assignment Vocal feedback regarding how to fix it (use the same ball again 


and ignore the warning) 


For this lesson, atom selection 1s a critical task sequence. Users should be protected from 
making errors. First of all, the user has to confirm that the atom 1s the one s (he) wants to select 
based on the augmentation. Second, this 1s done after the user input by providing a multimodal 
feedback that is targeting both the visual and auditory channels. Third, this is done by warning 
the user if the atom was already selected. Then, the user should be able to fix a selection error. 
This is done by repeating the selection using a ball of the same colour and ignoring the warning. 
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This example shows the importance of the immediate feedback (ergonomic criterion, see Bastien 
and Scapin, 1993) for user guidance. 

In order to be aware and avoid usability problems, the template presented in Table 4 
should be specified for each unit task. 


Conclusions 


AR technologies are expensive and require a lot of research and design effort to develop 
visualization and rendering software. A specific feature is the mix of traditional AR techniques 
with specific (dedicated) interaction techniques used to integrate domain specific real objects. 
In order to improve the ergonomic quality of e-learning applications, a user-centred approach 
is needed from scenario specification to implementation and evaluation. Using a task-based 
specification, incorporates human factors early in the design process, facilitates communication 
between designers and evaluators, and helps in fixing usability problems. 

The quality of user interfaces is a trade-off between utility and usability. A user-centred 
approach to design and evaluation could significantly improve the ergonomic quality of 
e-learning applications. This means, to focus on users’ tasks and goals. In this paper a task- 
based approach has been presented, that is, using the CTTE task modelling tool together with 
a tabular description of tasks and an analysis of potential usability problems. The approach is 
useful for both developers and evaluators. 

AR-based e-learning applications are featuring complex tasks that require both domain 
knowledge (e.g. understanding of Chemistry concepts) and specific knowledge of how to interact 
with an AR platform. Although the interaction is intuitive, the mix of real and virtual worlds 
may create additional difficulties for the learner as regarding the ease of use. The task based 
approach presented in this paper enables designers to better understand users’ needs and avoid 
potential usability problems that could undermine the ergonomic quality and, consequently, the 
outcomes of AR-based learning. 
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